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Abstract  The  NOAA  Great  Lakes  Operational  Forecast 
System  (GLOFS)  uses  near-real-time  atmospheric  observa¬ 
tions  and  numerical  weather  prediction  forecast  guidance  to 
produce  three-dimensional  forecasts  of  water  temperature 
and  currents,  and  two-dimensional  forecasts  of  water  levels 
of  the  Great  Lakes.  This  system,  originally  called  the  Great 
Lakes  forecasting  system  (GLFS),  was  developed  at  The 
Ohio  State  University  and  NOAA’s  Great  Lakes  Environ¬ 
mental  Research  Laboratory  (GLERL)  in  1989.  In  1996,  a 
workstation  version  of  the  GLFS  was  ported  to  GLERL  to 
generate  semi-operational  nowcasts  and  forecasts  daily.  In 
2004,  GLFS  went  through  rigorous  skill  assessment  and  was 
transitioned  to  the  National  Ocean  Service  (NOS)  Center  for 
Operational  Oceanographic  Products  and  Services  (CO-OPS) 
in  Silver  Spring,  MD.  GLOFS  has  been  making  operational 
nowcasts  and  forecasts  at  CO-OPS  since  September  30, 2005. 
Hindcast,  nowcast,  and  forecast  evaluations  using  the  NOS- 
developed  skill  assessment  software  tool  indicated  both 
surface  water  levels  and  temperature  predictions  passed  the 


Responsible  Editor:  Tal  Ezer 

This  article  is  part  of  the  Topical  Collection  on  2nd  International 
Workshop  on  Modelling  the  Ocean  2010 

P.  Y.  Chu  (El) 

Naval  Research  Laboratory, 

Stennis  Space  Center,  MS,  USA 
e-mail:  Philip.chu@nrlssc.navy.mil 

J.  G.  W.  Kelley 
NOAA/NOS/CSDL, 

Silver  Spring,  MD,  USA 

G.  V.  Mott  •  A.  Zhang 
NOAA/NOS/CO-OPS, 

Silver  Spring,  MD,  USA 

G.  A.  Lang 
NOAA/OAR/GLERL, 

Ann  Arbor,  MI,  USA 


NOS  specified  criteria  at  a  majority  of  the  validation  locations 
with  relatively  low  root  mean  square  error  (4-8  cm  for  water 
levels  and  0.5  to  1°C  for  surface  water  temperatures).  The 
difficulty  of  accurately  simulating  seiches  generated  by 
storms  (in  particular  in  shallow  lakes  like  Lake  Erie)  remains 
a  major  source  of  error  in  water  level  prediction  and  should  be 
addressed  in  future  improvements  of  the  forecast  system. 

Keywords  Numerical  modeling  •  Lake  forecasts  •  Coastal 
nowcast/forecast  lake  modeling  system 

1  Introduction 

The  Great  Lakes  of  North  America  is  the  largest  fresh  water 
body  in  the  world  with  a  surface  area  of  246,000  km2  and  a 
volume  of  22,684  km3.  It  consists  of  five  large  lakes  (Lakes 
Superior,  Huron,  Michigan,  Erie,  and  Ontario)  and  one  small 
lake  (Lake  St.  Clair),  as  shown  in  Fig.  1.  It  contains  more 
than  20%  of  the  world's  fresh  water  reserves,  is  shared  by  the 
USA  and  Canada,  and  supports  a  population  of  30  million 
along  its  perimeter  regions.  It  created  and  continues  to  create 
tremendous  economic,  commercial,  and  recreational  values 
and  at  the  same  time  is  undergoing  heavy  stress. 

The  Great  Lakes  Forecasting  System  (GLFS)  was 
developed  by  researchers  at  The  Ohio  State  University 
(OSU)  and  NOAA's  Great  Lakes  Environmental  Research 
Laboratory  (GLERL)  in  the  late  1980s  to  provide  nowcast 
and  forecast  guidance  of  water  levels,  water  temperatures, 
waves  and  currents  of  the  five  Great  Lakes.  The  main  uses 
for  those  predicted  variables  are  (1)  hazard  warning  and 
avoidance,  (2)  enhancement  of  recreational  and  commercial 
activities,  (3)  scenario  test  and  risk  assessment,  and  (4) 
natural  resources  preservation  and  decision  making. 

GLFS  used  the  Princeton  Ocean  Model  (POM)  (Blumberg 
and  Mellor  1987)  and  the  GLERL-Donelan  wave  model 
(Schwab  et  al.  1984)  to  predict  three-dimensional  (3-D) 
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Fig.  1  Great  Lakes  map 


temperature,  currents  and  two-dimensional  (2-D)  water 
levels,  and  waves  of  the  Great  Lakes.  The  first  3-D 
nowcast  for  the  Great  Lakes  was  made  in  1 992  at  the  Ohio 
Supercomputer  Center  at  OSU  (Yen  et  al.  1994;  Schwab 
and  Bedford  1994).  Twice  per  day  forecasts  were  made  for 
Great  Lakes  starting  in  1995  (Schwab  and  Bedford  1996). 
In  1996,  GLFS  was  ported  to  GLERL  in  Ann  Arbor,  MI, 
and  the  workstation  version  of  the  system  was  named  the 
Great  Lakes  Coastal  Forecast  System  (GLCFS).  GLCFS 
generates  nowcasts  four  times  per  day,  and  60-h  forecast 
guidance  twice  per  day  and  has  been  running  in  a  semi- 
operational  mode  since  February  1997  (Schwab  et  al. 
1999).  GLFS  was  recognized  as  the  first  US  coastal 
forecasting  system  to  make  routine  real-time  predictions  of 
currents,  temperatures,  and  key  trace  constituents  in  2001 
by  the  American  Meteorological  Society. 

In  2004,  the  hydrodynamic  model  code  of  GLCFS  was 
transitioned  to  the  National  Ocean  Service  (NOS)  Center  for 
Operational  Oceanographic  Products  and  Services  (CO-OPS) 
in  Silver  Spring,  MD.  GLCFS  was  reconfigured  to  mn  in  the 
NOS  Coastal  Ocean  Modeling  Framework  (COMF)  and  to  use 


surface  meteorological  observations  from  the  NOS  Operational 
Data  Acquisition  and  Archive  System  (ODAAS).  At  this  stage, 
the  system  went  through  significant  improvements,  upgrades, 
skill  assessment,  and  documentation,  and  the  new  system  was 
named  the  Great  Lakes  Operational  Forecast  System  (GLOFS) 
to  reflect  its  operational  status.  GLOFS  has  been  making 
operational  nowcasts  and  forecasts  at  CO-OPS  since  September 
30,  2005  and  was  the  first  NOS  forecast  system  to  be 
implemented  for  non-tidal  water  bodies.  This  article  describes 
the  model  development,  implementation,  and  skill  assessment 
of  the  Great  Lakes  operational  forecast  system. 

2  System  overview 

This  section  provides  a  brief  description  of  the  numerical 
hydrodynamic  model  used  by  GLOFS.  A  flowchart  of  this 
forecasting  system  is  shown  in  Fig.  2.  Similar  descriptions  of 
the  model  as  it  has  been  applied  to  Great  Lakes  have  been 
given  by  Kuan  (1995),  Kelley  (1995),  Kelley  et  al.  (1998), 
Hoch  (1997),  Chu  (1998),  and  O’Connor  et  al.  (1999). 
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Fig.  2  GLFS  flowchart 
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ANALYSIS 


2.1  Description  of  model 

The  core  numerical  model  in  GLOFS  is  the  POM 
developed  by  (Blumberg  and  Mellor  1987;  Mellor  2004). 
The  model  is  a  fully  three-dimensional,  non-linear  primitive 
equation  coastal  ocean  circulation  model,  with  a  second 
order  Mellor- Yamada  turbulence  closure  scheme  to  provide 
parameterization  of  vertical  mixing  processes.  The  model 
solves  the  continuity,  momentum,  and  conservation  equa¬ 
tions  simultaneously  in  an  iterative  fashion,  and  the 
resulting  predictive  variables  are  free  upper  surface  eleva¬ 
tion,  full  three-dimensional  velocity  and  temperature  fields, 
turbulence  kinetic  energy,  and  turbulence  macroscale.  Other 
main  features  of  the  model  include  terrain  following 
coordinate  in  the  vertical  (sigma  coordinate),  finite  differ¬ 
ence  numerical  scheme,  Boussinesq  and  hydrostatic  ap¬ 
proximation,  and  mode  splitting  technique.  Several 
modifications  have  been  made  by  OSU  and  GLERL 
researchers  for  use  in  the  Great  Lakes  (Bedford  and  Schwab 
1991;  O’Connor  and  Schwab  1993).  Each  of  the  Great 


Lakes  is  treated  as  an  enclosed  basin.  Therefore,  there  are 
no  inflow/outflow  boundary  conditions  and  no  fluid 
exchange  between  the  lake  and  its  tributaries. 

2.2  Grid  domain 

The  model  domain  for  Great  Lakes  consists  of  a  rectangular 
grid  with  a  5  km  horizontal  resolution  in  both  the  x-  and  y- 
directions  (except  for  Lake  Superior,  which  uses  10  km). 
The  bottom  topography  for  the  domain  is  based  on 
GLERL ’s  2-km  digital  bathymetry  data  compiled  by 
Schwab  and  Sellers  (1980),  but  slightly  smoothed  to 
minimize  noise.  The  Lake  Erie  domain  has  1,944  grid 
points  with  8 1  points  in  the  x-direction  and  24  points  in  the 
j-direction  with  a  27.33°  counterclockwise  rotation  so  that 
the  x-coordinate  is  along  the  longitudinal  axis  of  the  lake 
and  the  y- axis  is  across  the  lake  (Fig.  3).  Lake  Erie  has  1 1 
sigma  layers  in  the  vertical,  while  the  other  four  lakes  have 
20  layers.  Table  1  shows  the  grid  dimensions  for  all  five  of 
the  Great  Lakes  models. 


Fig.  3  Lake  Erie  grid  and  NOS 
gauge  stations 
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Table  1  Configurations  of  numerical  grid  domains  for  each  lake 


Lake 

Gird  size  (IXJ) 

Spatial  resolution  (km) 

Vertical 

layers 

Erie 

81x24 

5 

11 

Michigan 

53x102 

5 

20 

Ontario 

61x25 

5 

20 

Huron 

81x75 

5 

20 

Superior 

61x30 

10 

20 

2.3  The  data 

Two  types  of  input  forcing  are  required  by  the  system  to 
make  water  level,  temperature,  and  current  predictions: 
wind  stress  and  surface  heat  flux.  Wind  stress  u-  and  v- 
components  are  computed  from  surface  wind  speed  and 
directions,  while  heat  flux  is  estimated  from  wind  speed,  air 
temperature,  dew  point  temperature,  and  cloud  cover.  The 
main  difference  between  the  nowcast  and  forecast  guidance 
is  the  data  source:  The  nowcast  relies  on  surface  meteoro¬ 
logical  observations  (Fig.  4)  while  the  forecast  uses  the 
meteorological  forecasts  from  National  Weather  Service 
(NWS)  gridded  weather  forecasts  or  forecast  guidance  from 
the  NWS  numerical  weather  prediction  model.  The  obser¬ 


vations  consist  of  data  from  a  variety  of  sources  such  as  the 
Automated  Surface  Observing  System,  Coastal-Marine 
Automated  Network,  NOS  National  Water  Level  Observing 
Network  (NWLON),  and  NDBC’s  and  Environment 
Canada’s  buoys.  The  gridded  surface  weather  forecasts 
are  from  the  NWS  National  Digital  Forecast  Database 
(NDFD)  with  a  5 -km  spatial  resolution  and  the  NWS/ 
National  Centers  for  Environmental  Prediction  (NCEP) 
North  American  Mesoscale  (NAM)  Model  with  a  12-km 
resolution.  Both  are  available  from  NOS’  Operational 
Data  Acquisition  and  Archive  System  (ODAAS),  and 
NWS/NCEP  observational  “data  tanks”,  located  on  the 
NOAA  Central  Computer  Systems  (CCS). 

2.4  Nowcast  cycle 

The  nowcast  cycle  of  GLOFS  is  run  hourly  at  NOS  to 
generate  updated  hourly  nowcasts  of  3-D  water  temper¬ 
atures  and  currents  and  2-D  water  levels.  The  initial 
conditions  for  the  nowcast  cycle  are  provided  by  the  end 
of  the  previous  hour’s  nowcast  cycle.  The  nowcast  cycle  is 
forced  by  gridded  surface  meteorological  analyses  valid  at 
two  times,  1  h  prior  to  the  time  of  the  nowcast  and  the 
current  time  of  the  nowcast.  The  gridded  surface  meteoro¬ 
logical  analyses  are  generated  by  interpolating  surface 
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Fig.  4  Map  depicting  locations  of  Great  Lakes  meteorological  stations 
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observations  using  the  natural  neighbor  interpolation 
technique  (Sambridge  et  al.  1995). 

The  surface  wind,  air  temperature,  and  other  meteoro¬ 
logical  parameter  observations  are  adjusted  to  a  10-m 
height,  corrected  for  stability  and  overwater  conditions 
before  being  interpolated.  Both  adjustments  use  the 
previous  day’s  lake  average  water  temperature  from 
GLERL’s  Great  Lakes  Surface  Environmental  Analysis 
(GLSEA).  The  GLSEA  temperature  analysis  is  generated 
using  sea  surface  temperature  retrievals  derived  from  the 
Advanced  Very  High  Resolution  Radiometer  onboard 
NOAA’s  polar-orbiter  weather  satellites. 

The  gridded  surface  wind  fields  are  then  used  to 
calculate  wind  stress  at  each  model  grid  point.  The 
surface  meteorological  fields  along  with  lake  surface 
water  temperature  predictions  are  used  by  a  heat  flux 
scheme  (McCormick  and  Meadows  1988)  to  estimate  the 
net  rate  of  heat  transfer  for  the  lake  at  each  grid  point. 
Additional  information  on  the  wind  stress  and  heat  flux 
implementation  schemes  can  be  found  in  Kelley  (1995) 
and  Chu  (1998). 

2.5  Forecast  cycle 

The  forecast  cycle  of  GLOFS  is  run  four  times  per  day  to 
generate  up  to  a  36-h  forecast  guidance  of  the  3-D  and  2-D 
state  of  Great  Lakes.  The  forecast  cycle  uses  the  most 
recent  nowcast  as  its  initial  conditions.  The  surface 
meteorological  forcing  is  provided  by  the  latest  forecast  of 
surface  u-  and  v-wind  components  and  surface  air  temper¬ 
ature  from  the  00,  06,  12,  or  18  UTC  forecast  cycles  from 
the  NDFD  or  the  NAM  model.  Prior  to  April  2007,  NAM, 
which  uses  the  Weather  Research  and  Forecast  Model  as  its 
core  model,  was  the  main  input  for  GLOFS  forecast  cycles. 
After  April  2007,  NDFD  was  the  primary  input  and  NAM 
served  as  a  source  of  backup  forcing. 

3  Skill  assessments 

3.1  NOS  evaluation  standard  and  skill  assessment  software 

In  order  to  ensure  the  nowcast/forecast  systems  developed 
and  implemented  at  NOS  are  done  so  in  a  scientifically 
sound  and  operationally  robust  way,  standards  for  evaluat¬ 
ing  such  modeling  systems  (Hess  et  al.  2003)  have  been 
established  and  skill  assessment  software  have  been 
developed  at  Coast  Survey  Development  Laboratory 
(CSDL)  (Zhang  et  al.  2006).  The  skill  assessment  software 
computes  various  statistic  measures  after  collecting  files 
containing  observation  records,  computed  hindcast,  now- 
cast,  and  forecast  variables  such  as  water  levels,  temper¬ 
atures,  and  currents.  All  data  are  processed,  and  the  skill 


assessment  results  are  tabulated  for  each  location.  This 
section  describes  the  standard,  statistic  metrics,  and  GLOFS 
skill  assessment  results. 

According  to  NOS  standards,  hydrodynamic  models  are 
required  to  be  executed  and  the  results  evaluated  under 
three  scenarios  before  declared  operational:  (1)  hindcast 
scenario,  (2)  semi-operational  nowcast,  and  (3)  semi- 
operational  forecast.  In  this  context,  the  hindcast  scenario, 
model  forcing  is  based  on  historical,  quality-checked,  best 
available  gap-filled  data.  The  model  result  time  series  can 
be  compared  with  the  available  observations.  In  semi- 
operational  nowcast  scenario,  the  model  forcing  is  based  on 
real-time  observed  values  where  there  may  be  missing  or 
incomplete  records.  In  semi-operational  forecast  scenario, 
the  model  forcing  is  based  on  forecast  values  from  weather 
forecast  models,  even  though  some  data  could  be  missing 
or  delayed.  In  evaluating  GLOFS,  NOS  took  advantage  of 
previous  evaluations  done  by  researchers  at  OSU  and 
GLERL  to  fulfill  the  hindcast  scenario  requirements  and 
utilized  the  nowcasts  and  forecasts  routinely  produced  by 
GLERL  to  fulfill  the  semi-operational  nowcast  and  forecast 
scenarios. 

3.2  Statistic  metrics 

The  standard  NOS  model  assessment  statistics  suite  include 
root  mean  square  error  (RMSE),  mean  algebraic  error 
(MAE),  series  mean  (SM),  standard  deviation  (SD),  central 
frequency  (CF),  negative  outlier  frequency  (NOF),  and 
positive  outlier  frequency  (POF).  Their  definitions  and 
formulations  are  given  in  Table  2. 

In  addition  to  the  above  statistical  measurements,  two 
additional  indices,  index  of  agreement  (IOA)  and  skill 
score,  were  also  computed  in  the  hindcast  evaluation.  The 
IOA  is  a  relative  measure  reflecting  the  degree  to  which  the 
observed  variable  is  estimated  by  the  simulated  variable 
(Willmott  1981).  IOA  is  related  to  the  RMSE  and  is  defined 
as  IOA=  1  -  {n  x  RMSE2)/PE,  where  PE  is  the  potential  error 
and  IOA  ranges  from  0  to  1  with  1  being  perfect  agreement. 
Skill  score  is  a  non-parametric-based  statistic  developed  by 
Dingman  and  Bedford  (1986)  to  assess  the  model  credibility. 


Table  2  NOS  skill  assessment  statistics  (from  Hess  et  al.  2003) 


Variable 

Equation 

Mean  algebraic  error 

e = prediction  -  observation 

( e  or  MAE) 

N 

Series  mean 

y  =  bT,yi- 

(SM) 

,_1  /  N 

Root  mean 

RMSE  =,4Ee?- 

square  error 

V  i=1 

(RMSE) 

Standard  deviation 

SD=  iVrE(e-e)2 

(SD) 

V  >=i 

Springer 


1310 


Ocean  Dynamics  (2011)  61:1305-1316 


The  rule  of  the  skill  score  for  water  levels  is  that  one  point  is 
deducted  from  a  scale  of  10  for  every  0.05  m  difference 
between  the  observed  and  computed  values  with  a  minimum 
score  of  0  when  the  difference  is  greater  than  0.5  m.  The 
same  analogy  is  applied  for  the  temperature  where  one  point 
is  deducted  for  every  0.5°C  model-data  difference. 

Since  the  NOS  skill  assessment  software  was  designed  to 
evaluate  model  skills  on  water  levels,  temperature  and 
currents  in  tidally  dominated  regions,  several  modifications 
were  made  for  the  non-tidal,  fresh  water  Great  Lakes:  (1)  Tidal 
evaluations  and  harmonic  analysis  in  the  water  level  were  not 
computed,  extreme  high-low  water  level  events  were  evalu¬ 
ated  instead,  and  (2)  the  acceptance  criteria  for  water 
temperature  has  been  adjusted  for  the  Great  Lakes  region. 
The  NOS  sets  an  acceptable  error  of  7.7°C  and  7.5  cm  (3  in.) 
in  evaluating  water  temperature  and  water  level  predictions  in 
tidal  regions.  Since  the  Great  Lakes  are  considered  non-tidal, 
there  is  no  preset  standard  for  lake  temperature  prediction.  A 
3°C  criterion  for  water  temperature  was  suggested  by  Dr. 
David  Schwab  of  NOAA/GLERL  based  on  the  experience  of 
running  the  Great  Lakes  Coastal  Forecasting  System  and  input 
from  the  Great  Lakes  user  community  (Schwab,  personal 
communication  2004). 

3.3  Hindcast  skill  assessment 

In  order  to  fulfill  the  NOS  hindcast  requirement,  several 
previous  research  efforts  and  evaluations  were  used:  for 
Lake  Erie,  a  comprehensive  evaluation  of  water  levels, 
temperature,  and  currents  (both  surface  and  subsurface) 
performed  by  Kuan  (1995)  and  Kuan  et  al.  (1995)  served  as 
the  basis  for  the  hindcast  scenario  assessment,  and  the 
validation  results  were  summarized  in  the  NOAA  technical 
report  (Chu  et  al.  2007).  The  Lake  Michigan  portion  of  the 
hindcast  was  done  using  the  data  and  modeling  results  from 
Lake  Michigan  Mass  Balance  Study  (Schwab  and  Beletsky 
1998),  and  the  skill  assessment  was  summarized  in  Kelley 
et  al.  (2007a,  b).  A  30-year  Lake  Ontario  thermal  structure 
reconstruction  was  used  to  satisfy  the  Lake  Ontario 
hindcast  requirement. 

Lake  Erie,  due  to  its  shallowness  (20  m  average  depth) 
and  SW-NE  orientation,  responds  quickly  to  the  passage  of 
weather  systems  (Figs.  5  and  6).  The  wind-driven  water 
level  fluctuations  (seiche)  and  thermal  response  (Schertzer 
et  al.  1987)  are  much  more  significant  than  that  of  the  other 
four  lakes  and  hence  pose  a  greater  challenge  in  data-model 
comparison.  The  seiches  cause  a  sudden  opposite  water 
level  change  at  the  two  ends  of  the  lake,  as  seen  for  most 
storm  events  in  Buffalo  (Fig.  5)  and  Toledo  (Fig.  6). 
Therefore  in  this  article,  all  the  statistic  analyses  and  plots 
use  Lake  Erie  as  example.  Detailed  skill  assessment  and 
statistics  for  other  lakes  are  presented  in  individual  skill 
assessment  technical  reports. 
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Fig.  5  Observed  vs.  simulated  water  levels  at  NOS  NWLON  stations 
in  Buffalo,  NY  during  the  period  1  April  to  3 1  December  2004 


Hindcasts  of  Lake  Erie  water  levels  at  Buffalo,  NY; 
Cleveland,  OH;  and  Toledo,  OH  were  compared  with 
observed  values  at  corresponding  NOS  water  level  gauges. 
RMSE  and  the  amplitude  skill  scores  are  summarized  in 
Table  3.  The  average  RMSE  for  water  elevations  during  the 
evaluation  period  (May  29  to  October  26,  1979)  at  Buffalo, 
Cleveland,  and  Toledo  were  4.82,  3.0,  and  6.18  cm, 
respectively.  The  amplitude  skill  scores  for  the  entire  test 
period  were  9.73,  9.91,  and  9.52  for  Buffalo,  Cleveland, 
and  Toledo,  respectively. 

An  evaluation  of  water  levels  on  a  seasonal  basis  was 
also  performed  during  the  heating,  stratified,  and  cooling 
seasons  (Table  4).  Heating  season  was  defined  from  May 
29  to  July  23.  The  stratified  season  began  on  July  24  and 
ended  on  September  6,  and  the  cooling  season  began  at  the 
end  of  stratified  season. 
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Fig.  6  Observed  vs.  simulated  water  levels  at  NOS  NWLON  stations 
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Table  3  Summary  of  Lake  Erie  water  level  hindcast  evaluation 
statistics  by  NOS  gauge  station  for  five  time  periods  in  1979 


Period 

NOS  station 

name 

RMSE 

(cm) 

Skill  score 
(0-10) 

Days  of 
year 

Calendar  days 

149-177 

5/29-6/28/79 

Buffalo 

4.05 

9.81 

149-177 

5/29-6/28/79 

Cleveland 

2.89 

9.93 

149-177 

5/29-6/28/79 

Toledo 

5.51 

9.61 

179-207 

6/28-7/28/79 

Buffalo 

3.84 

9.85 

179-207 

6/28-7/28/79 

Cleveland 

2.65 

9.93 

179-207 

6/28-7/28/79 

Toledo 

4.97 

9.64 

209-237 

7/28-8/27/79 

Buffalo 

4.54 

9.73 

209-237 

7/28-8/27/79 

Cleveland 

2.94 

9.90 

209-237 

7/28-8/27/79 

Toledo 

6.27 

9.51 

239-267 

8/27-9/26/79 

Buffalo 

4.25 

9.77 

239-267 

8/27-9/26/79 

Cleveland 

2.37 

9.96 

239-267 

8/27-9/26/79 

Toledo 

5.73 

9.57 

269-297 

9/26-10/26/79 

Buffalo 

6.80 

9.49 

269-297 

9/26-10/26/79 

Cleveland 

3.93 

9.82 

269-297 

9/26-10/26/79 

Toledo 

7.99 

9.27 

The  amplitude  skill  scores  and  IOA  indicate  that  the 
water  level  simulations  are  good  for  all  seasons.  However, 
both  tables  show  higher  RMSE  in  the  cooling  season  and 
Toledo  tends  to  have  larger  errors  than  other  two  stations. 
The  average  RMSE  between  the  computed  and  observa¬ 
tions  were  4.01,  4.23,  and  5.61  cm  at  Buffalo,  Cleveland, 
and  Toledo,  respectively,  with  the  corresponding  average 
skill  scores  of  9.97,  9.76,  and  9.60  for  the  heating, 
stratified,  and  cooling  seasons,  respectively. 

To  evaluate  model  performance  of  lake  surface  temper¬ 
atures,  field  measurements  at  six  Canada  Centre  for  Inland 
Water  (CCIW)  meteorological  buoys  were  used  for  com¬ 
parison.  RMSE,  IOA,  and  skill  scores  during  May  through 
October  1979  are  summarized  in  Table  5. 


Table  4  Seasonal  evaluation  statistics  for  Lake  Erie  water  level 
simulations  during  heating,  stratified,  and  cooling  seasons  in  1979 

Season 

NOS  station 

RMSE 

(cm) 

IOA 

(0-1.0) 

Skill  score 
(0-10) 

Heating 

Buffalo 

3.95 

0.89 

9.83 

Heating 

Cleveland 

2.79 

0.72 

9.93 

Heating 

Toledo 

5.28 

0.92 

9.62 

Stratified 

Buffalo 

4.27 

0.96 

9.77 

Stratified 

Cleveland 

2.74 

0.88 

9.92 

Stratified 

Toledo 

5.69 

0.94 

9.58 

Cooling 

Buffalo 

6.01 

0.95 

9.58 

Cooling 

Cleveland 

3.41 

0.94 

9.87 

Cooling 

Toledo 

7.41 

0.96 

9.35 

Table  5  Evaluation  statistics  for  surface  water  temperature  simula¬ 
tions  during  1979  at  six  CCIW  buoys 

CCIW  buoy  ID 

RMSE  (°C) 

IOA 

Skill  score 

NWRI19A 

1.16 

0.97 

8.66 

NWRI24A 

1.07 

0.97 

8.65 

NWRI26A 

0.92 

0.98 

8.98 

NWRI42A 

1.06 

0.97 

8.69 

NWRI46A 

0.90 

0.98 

9.04 

NWRI47A 

0.87 

0.98 

9.02 

Throughout  the  entire  evaluation  period,  the  Lake  Erie 
temperature  simulations  matched  well  with  the  observed 
data  at  all  six  buoys.  Slight  differences  existed  between  the 
observed  and  computed  values.  From  Table  5,  the  average 
skill  score  was  8.84  and  the  average  IOA  was  0.97;  both 
indices  showed  good  agreement  between  the  predicted  and 
observed  values.  The  average  RMSE  was  less  than  1°C 
which  is  also  an  indication  that  the  model  simulated  lake 
surface  temperature  very  well.  Statistics  due  to  the  seasonal 
variations  are  summarized  in  Table  6. 

In  general,  water  surface  elevation  hindcasts  matched 
well  in  magnitude  with  the  corresponding  observed  data  by 
picking  up  almost  every  single  significant  peak  appearing 
in  the  observed  water  levels.  The  small  RMSE  and  high 
IOA  indicated  that  the  model  is  accurate  in  simulating 
water  surface  elevation.  The  modeling  system  performed 
equally  well  in  simulating  surface  water  temperature.  The 
computed  values  followed  the  trend  of  the  observed  data 
closely.  The  performance  for  the  cooling  season  was  a  bit 
worse  than  the  other  two  seasons  and  depended  upon  buoy 
locations.  During  the  cooling  season,  surface  water  temper¬ 
atures  were  consistently  over  predicted  by  1.1  °C.  As  seen 
in  Table  6,  the  model  had  better  skill,  in  terms  of  RMSE 
and  skill  scores,  for  the  heating  and  stratified  seasons  than 
the  cooling  season. 

3.4  Semi-operational  nowcast  skill  assessment 

The  skill  assessment  of  the  GLOFS  semi-operational  now¬ 
casts  was  performed  using  GLERL’s  archived  nowcasts.  Time 
series  of  hourly  water  levels  and  surface  temperature  nowcasts 
were  compared  to  observations  from  NOS  NWLON  water 

Table  6  Average  seasonal  analysis  for  surface  water  temperature 
simulations 

Heating  season  Stratified  season  Cooling  season 

RMSE  IOA  Skill  RMSE  IOA  Skill  RMSE  IOA  Skill 

(°C)  score  (°C)  score  (°C)  score 

1.05  0.97  8.81  0.80  0.88  9.20  1.10  0.95  8.55 
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Table  7  Summary  of  skill  assessment  statistics  of  semi-operational  nowcasts  of  hourly  water  levels  at  eight  NOS  NWLON  stations  in  Lake  Erie 
for  the  period  15  April  to  17  December  2004 


Statistic,  acceptable  error  [], 
and  units  () 

Buffalo, 

NY 

Sturgeon  Point, 
NY 

Erie, 

PA 

Fairport, 

OH 

Cleveland, 

OH 

Marblehead, 

OH 

Toledo, 

OH 

Fermi  Power 
Plant,  MI 

NOS  acceptance 
criteria 

Mean  algebraic  error  (m) 

0.026 

0.034 

0.008 

-0.031 

0.008 

0.000 

-0.029 

-0.005 

0.075 

RMSE  (m) 

0.080 

0.076 

0.045 

0.044 

0.040 

0.050 

0.080 

0.065 

na 

SD  (m) 

0.076 

0.068 

0.044 

0.031 

0.040 

0.050 

0.075 

0.065 

na 

NOF  [2  x  15cm]  (%) 

0.8 

0.5 

0.0 

0.0 

0.0 

0.0 

0.3 

0.1 

<  1% 

CF  [15  cm]  (%) 

95.6 

96.4 

98.9 

99.7 

99.1 

98.4 

94.0 

96.6 

>  90% 

POF  [2x15  cm]  (%) 

0.0 

0.0 

0.0 

0.0 

0.1 

0.0 

0.1 

0.1 

<  1% 

A  total  of  5,832  nowcast  records  were  used  in  the  assessment 


level  stations  and  NDBC  buoy  temperature  records  along  the 
Great  Lakes  for  the  period  from  April  to  December  2004. 
Time  series  plots  of  the  water  level  nowcasts  vs.  observations 
at  Buffalo  and  Toledo  are  given  in  Figs.  5  and  6.  Both  figures 
clearly  show  the  drawdown  at  Toledo  and  high  setup  at 
Buffalo  (the  result  of  wind-driven  lake  seiches  dynamics) 
during  the  strong  storm  events  between  Julian  day  290  and 
350.  The  skill  statistics  for  the  Lake  Erie  water  level 
nowcasts  to  at  eight  NOS  gauges  during  2004  are  presented 
together  in  Table  7  along  with  the  NOS  acceptance  criteria. 
The  hourly  nowcasts  passed  NOS  acceptance  criteria  at  all 
eight  NOS  gauge  locations. 

The  MAE  differences  ranged  between  -2.9  and  +3.4  cm 
and  the  RMSE  ranged  between  4  and  8  cm.  The  largest 
RMSE  were  at  Toledo,  OH  and  Buffalo,  NY  (Fig.  3)  where 
nowcasts  under  predicted  at  Toledo  and  over  predicted  at 
Buffalo.  Toledo  and  Buffalo,  located  at  the  extreme  SW  and 


Fig.  7  Time  series  plot  of  semi-operational  nowcast  and  forecast 
guidance  of  surface  water  temperatures  (degrees  Celsius)  vs.  observa¬ 
tions  at  NDBC  buoy  45005  (Western  Lake  Erie)  for  the  period  21 
April  to  8  December  2004 


NE  ends  of  the  lakes,  respectively,  experience  the  greatest 
water  level  variability  and  are  the  most  difficult  locations  to 
predict.  The  differences  at  gauge  locations  to  the  north 
(Erie,  PA)  and  south  (Cleveland,  OH)  of  the  lake  were  less 
than  1  cm. 

The  evaluation  of  surface  water  temperatures  nowcasts  was 
based  on  comparisons  of  time  series  of  NOAA/NDBC  buoy 
data  vs.  model-predicted  temperatures  from  April  to 
December  2004.  A  time  series  plot  of  the  nowcasts  vs. 
observations  at  the  buoy  45005  is  given  in  Fig.  7.  The  time 
series  plot  indicates  that  the  nowcasts  were  in  close 
agreement  with  observations  (+0.5  to  +1°C)  from  mid- 
April  until  early  May,  but  then  began  to  deviate  from  the 
observations  by  +1  to  +2°C  until  late  May  (JD150).  After 
that,  the  nowcasts  differ  from  observations  by  +0.5°C  until 
mid- August  (JD230).  The  nowcasts  then  deviated  by  +1 
to  +2°C  until  early  October  (JD280).  During  the  remaining 
days  of  autumn  through  the  mid-December,  the  nowcasts 
generally  differed  from  observations  by  +0.5°C. 

The  skill  statistics  for  predicted  hourly  surface  water 
temperature  at  NDBC  buoy  45005  are  given  in  Table  8 
along  with  the  NOS  acceptance  criteria.  The  hourly  water 
temperature  nowcasts  at  the  buoy  did  pass  the  NOS 

Table  8  Summary  of  skill  assessment  statistics  of  the  semi- 
operational  hourly  nowcasts  of  surface  water  temperatures  at  the 
NDBC  buoy  45005  in  Lake  Erie  for  the  period  from  mid-April  to 
early  December  2004 


Time  period,  statistic,  acceptable 
error  [],  and  units  () 

45005  West  Erie 
(N=  5566) 

NOS  acceptance 
criteria 

Time  Period  (days) 

202 

365 

Mean  Difference  (°C) 

0.951 

na 

RMSE  (°C) 

1.292 

3° 

SD  (°C) 

0.875 

na 

NOF  [2x3°C]  (%) 

0.0 

<1% 

CF  [3°C]  (%) 

98.7 

>90% 

POF  [2x3°C]  (%) 

0.0 

<1% 
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Table  9  Summary  of  skill  assessment  statistics  of  semi-operational  24-hr  forecast  guidance  of  hourly  water  levels  at  NOS  NWLON  stations  in 
Lake  Erie  for  the  period  15  April  to  17  December  2004 


Statistic,  acceptable  Buffalo,  NY  Sturgeon  Erie,  PA  Fairport,  OH  Cleveland,  Marblehead,  Toledo,  OH  Fermi  Power  NOS 


error  [],  and  units  () 

(A=490) 

Point,  NY 
(A=490) 

o 

On 

'nT 

o 

o 

o\ 

'nT 

K 

o 

OH  (A=473) 

oo 

Plant,  MI 
(A=477) 

acceptance 

criteria 

Mean  algebraic 

0.036 

0.044 

0.017 

-0.030 

0.006 

-0.008 

-0.034 

-0.016 

0.075 

error  (m) 

RMSE  (m) 

0.088 

0.084 

0.052 

0.044 

0.041 

0.065 

0.107 

0.086 

na 

SD  (m) 

0.080 

0.072 

0.050 

0.032 

0.041 

0.065 

0.102 

0.084 

na 

NOF  [2x15  cm]  (%) 

0.4 

0.4 

0.2 

0.0 

0.0 

0.4 

1.4 

0.8 

<1% 

CF  [15  cm]  (%) 

95.9 

96.1 

98.6 

99.4 

99.4 

97.3 

87.7 

93.9 

>90% 

POF  [2x15  cm]  (%) 

0.4 

0.6 

0.0 

0.0 

0.0 

0.2 

0.2 

0.2 

<1% 

Number  in  bold  indicates  that  the  statistic  did  not  pass  the  NOS  criteria 


acceptance  criteria  for  all  the  assessment  statistics.  The 
MAE  was  0.95°C,  and  the  RMSE  was  1.29°C. 

3.5  Semi-operational  forecast  skill  assessment 

For  the  assessment  of  the  semi-operational  forecast  scenario 
for  GLOFS,  archived  GLCFS  semi-operational  24- 
h  forecast  guidance  was  compared  to  water  level  observa¬ 
tions  from  NOS  NWLON  stations  from  April  to  December 
2004,  and  to  NDBC  buoy  data  from  April  to  November 
2004  for  the  surface  water  temperature  forecasts.  Water 
level  skill  assessment  statistics  for  the  semi-operational 
forecast  guidance  at  eight  NWLON  stations  along  with  the 
NOS  acceptance  criteria  are  given  in  Table  9. 

The  hourly  forecasts  passed  the  criteria  at  seven  of  the  eight 
gauges,  failing  only  at  Toledo  for  central  frequency  by  a  small 
margin.  The  MAE  values  ranged  between  -3  and  +4.4  cm, 
and  the  RMSE  ranged  between  4.1  cm  at  Cleveland  and 
10.7  cm  at  Toledo.  Similar  to  the  nowcasts,  the  greatest 
errors  were  at  Buffalo  and  Toledo,  located  at  the  extreme 
ends  of  the  lake.  The  forecasts  under  predicted  the  water 
levels  at  Toledo  and  over  predicted  the  levels  at  Buffalo. 
There  was  some  increase  in  the  RMSE  values  as  forecast 
projection  increased. 

Skill  assessment  statistics  for  semi-operational  forecast 
guidance  to  predict  hourly  lake  surface  water  temperatures 
at  NDBC  buoy  45005  along  with  the  NOS  acceptance 
criteria  are  tabulated  in  Table  10,  and  the  time  series  plots 
are  given  in  Fig.  7.  Similar  to  the  nowcasts,  the  semi- 
operational  forecast  guidance  was  in  close  agreement  to 
observations  (+0.5  to  +1°C)  from  mid- April  until  early 
May,  but  then  began  to  deviate  from  the  observations  by  +1 
to  +2°C  until  late  May.  After  that,  the  forecast  guidance 
differed  from  observations  by  +0.5°C  until  mid- August. 
The  guidance  then  deviated  by  +1  to  +2°C  until  early 
October.  During  the  remaining  days  of  autumn  through 
mid-December,  the  forecast  guidance  generally  differed 
from  observations  by  +0.5°C.  The  hourly  forecast  guidance 


at  the  buoy  passed  all  NOS  criteria.  The  MAE  was  0.7°C, 
and  the  RMSE  was  1.3°C.  The  MAE  and  RMSE  for  the 
forecast  guidance  were  slightly  lower  than  for  the  nowcasts 
(Table  8).  It  is  interesting  to  note  that  MAE  and  RMSE 
values  decreased  as  forecast  projection  increased  in  time. 
The  MAE  was  1.07°C  at  the  0-h  projection  and  reduced  to 
0.7 1°C  by  the  24-h  projection.  Both  statistics  suggest  that 
the  surface  heat  flux  is  being  overestimated  during  the 
nowcast  cycle  and  that  temperature  prediction  is  cooling  off 
during  the  forecast  cycle  when  there  is  no  surface  heat  flux 
input. 

For  water  levels,  both  hindcast  and  nowcast/forecast 
statistics  show  large  error  at  the  far  ends  of  Lake  Erie 
(Toledo  and  Buffalo).  Although  model  grid  resolution  and 
the  lack  of  tributary  input  (Maumee  and  Niagara  Rivers) 
could  be  potential  sources  of  error,  we  believe  the  seiche 
effect  is  the  main  source  as  RMSE  in  both  locations  are  also 
higher  during  the  cooling  season  when  storms  occur 
frequently. 

For  lake  surface  temperature,  both  nowcasts  and  fore¬ 
casts  indicated  a  warm  bias.  Possible  errors  are  (1) 
overestimation  of  surface  heat  flux,  (2)  lack  of  ice  module, 

Table  10  Summary  of  skill  assessment  statistics  for  semi-operational 
forecast  guidance  to  predict  surface  water  temperatures  24  h  in 
advance  at  NDBC  buoy  45005  in  Lake  Erie  during  the  period  from 
mid-April  to  early  November  2004 


Time  period,  statistic,  acceptable 
error  [],  and  units  () 

45005  (A=460) 

NOS  acceptance 
criteria 

Time  period 

202 

365  days 

Mean  algebraic  error  (°C) 

0.713 

na 

RMSE  (°C) 

1.306 

3° 

SD  (°C) 

1.095 

na 

NOF  [2x3°C]  (%) 

0.0 

<1% 

CF  [3°C]  (%) 

98.7 

>90% 

POF  [2x3°C]  (%) 

0.0 

<1% 
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Fig.  8  Example  of  LEOFS  map 
wind  fields 
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and  (3)  inadequate  model  vertical  resolution.  Potential 
solutions  include:  (1)  improve  the  surface  heat  flux 
formulation,  especially  the  cloud  cover  parameterization  in 
the  shortwave  and  longwave  radiation  components,  (2) 
incorporate  an  ice  module  with  better  thermodynamics  to 
better  account  for  the  total  lake  heat  content  and  distribu¬ 
tion,  and  (3)  increase  the  vertical  resolution  to  better  resolve 
the  thermocline. 

3.6  Improvements  during  the  transition  stage 

During  the  evolution  of  GLFS  to  GLCFS  and  the  transition 
into  GLOFS,  every  component  of  the  system  has  been 
upgraded  or  improved  to  a  certain  degree,  except  the  main 
POM  ocean  circulation  model  code.  The  initial  project 
goals  included  maintaining  the  model  accuracy,  enhancing 
the  system  robustness  and  reliability,  setting  a  standard  for 
the  future  NOS  operational  oceanographic  forecast  model¬ 
ing  systems  while  maintaining  a  consistent  “look-and-feel” 


of  GLOFS  products,  and  dissemination  procedures  for  end 
users.  Significant  improvements  include:  (1)  automate 
scripts  to  streamline  data  ingestion  and  output,  (2)  modify 
forecast  system  requirements  of  NOS’  Coastal  Ocean 
Modeling  Framework  (COMF)  to  enhance  system  reliabil¬ 
ity,  (3)  complete  unified  netCDF-based  file  I/O,  and  (4)  add 
new  error  checking,  gap  filling  procedures,  and  extend  the 
forecast  period  to  60  h. 

4  Summary  and  conclusion 

This  article  describes  the  historical  background,  develop¬ 
ment,  implementation,  transition,  and  skill  assessment  of 
the  NO  A  A  Great  Lakes  Operational  Forecast  System 
(GLOFS).  GLOFS  is  the  result  of  technology  transfer  of 
the  Great  Lakes  Forecasting  System  (GLFS)  and  the  Great 
Lakes  Coastal  Forecasting  System  (GLCFS)  from  The  Ohio 
State  University  (OSU)  and  NOAA’s  Great  Lakes  Environ- 


Fig.  9  Example  of  LEOFS  map 
current  fields 
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Fig.  10  Example  of  LEOFS 
map  water  levels 


Valid  at  1600  (ESI)  11/06/07 


mental  Research  Laboratory  (GLERL).  GLOFS  has  gone 
through  vigorous  testing,  skill  assessment,  and  thorough 
documentation  as  it  has  been  transitioned  from  research  to 
operations  at  NOS.  GLOFS  was  officially  implemented  and 
declared  operational  at  NOS/CO-OPS  on  September  30, 
2005. 

The  NOS  in-house  developed  skill  assessment  software 
was  used  to  evaluate  the  water  level  and  temperature  skill 
performance  of  the  GLOFS.  A  suite  of  statistics  was 
computed  for  water  levels  and  lake  surface  temperatures 
for  hindcast,  semi-operational  nowcast,  and  forecast  sce¬ 
narios.  Skill  assessment  results  indicated  that  GLOFS 
passed  the  NOS  7.5  cm  for  water  level  and  3°C  for  water 
temperature  acceptance  criteria  most  of  the  time  for 
majority  of  the  validation  locations.  Water  level  nowcasts 
tend  to  have  better  passing  rates  than  the  forecast  guidance; 
however,  surface  temperature  forecasts  have  slightly  lower 
RMSE  than  that  of  the  nowcast. 

During  the  transition  process,  the  model  code  and 
associated  programs  were  integrated  into  the  NOS  COMF 


modeling  framework  and  run  on  Linux  clusters  located  at 
the  NOAA  CO-OPS  division.  All  the  GLOFS  products, 
including  data  plots  and  maps  as  well  as  netCDF  files 
containing  formatted  model  results,  are  available  to  the 
general  public  at  http://tidesandcurrents.noaa.gov.  Exam¬ 
ples  of  Lake  Erie  Operational  Forecast  System  (LEOFS) 
map  products  are  given  in  Figs.  8,  9,  10,  and  11. 

Recently,  GLOFS  went  through  another  transition  when 
it  ported  to  NOAA’s  Centralized  Computing  System 
operated  by  NCEP  to  provide  a  robust  operational 
computational  environment  and  to  extend  its  forecast 
horizon  from  36  to  60  h.  GLOFS  became  operational  on 
the  CCS  on  December  7,  2010.  Future  improvements  of 
the  GLOFS  include  increasing  the  spatial  model  resolu¬ 
tion  from  5  to  2  km,  incorporating  an  ice  modeling 
component,  adding  tributary  information,  running  all  five 
lakes  as  a  single  modeling  system  by  connecting  all  the 
channels,  and  possibly  using  data  assimilation  approach 
to  improve  the  surface  heat  flux  estimation  and  water 
temperature  predictions. 


Fig.  11  Example  of  LEOFS 
map  surface  temperature 
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